Turn-alignment using eye-gaze and speech in conversational interaction
نویسندگان
چکیده
Spoken interactions are known for accurate timing and alignment between interlocutors: turn-taking and topic flow are managed in a manner that provides conversational fluency and smooth progress of the task. This paper studies the relation between the interlocutors’ eye-gaze and spoken utterances, and describes our experiments on turn alignment. We conducted classification experiments by Support Vector Machine on turn-taking using the features for dialogue act, eye-gaze, and speech prosody in conversation data. As a result, we demonstrated that eye-gaze features are important signals in turn management, and seem even more important than speech features when the intention of utterances is clear.
منابع مشابه
Incorporating Temporal and Semantic Information with Eye Gaze for Automatic Word Acquisition in Multimodal Conversational Systems
One major bottleneck in conversational systems is their incapability in interpreting unexpected user language inputs such as out-ofvocabulary words. To overcome this problem, conversational systems must be able to learn new words automatically during human machine conversation. Motivated by psycholinguistic findings on eye gaze and human language processing, we are developing techniques to inco...
متن کاملGaze, conversational agents and face-to-face communication
In this paper, we describe two series of experiments that examine audiovisual face-to-face interaction between naive human viewers and either a human interlocutor or a virtual conversational agent. The main objective is to analyze the interplay between speech activity and mutual gaze patterns during mediated face-to-face interactions. We first quantify the impact of deictic gaze patterns of our...
متن کاملNon-verbal Signals for Turn-taking and Feedback
This paper concerns non-verbal communication, and describes especially the use of eye-gaze to signal turn-taking and feedback in conversational settings. Eye-gaze supports smooth interaction by providing signals that the interlocutors interpret with respect to such conversational functions as taking turns and giving feedback. New possibilities to study the effect of eye-gaze on the interlocutor...
متن کاملMultimodal Communication in Computer-Mediated Map Task Scenarios
Multimodal communication involves the co-occurrence of different communicative channels, including speech, eye gaze and facial expressions. The questions addressed in this study are how these modalities correlate and how they are aligned to the discourse structure. The study focuses on a map task scenario whereby participants coordinate a route on a map, while their speech, eye gaze, face and t...
متن کاملCreation of Multimodal Corpus for Modeling Turn Management in Multi-party Conversations
Spoken interactions are known for accurate timing and alignment between interlocutors: turn-taking and topic flow are managed in a manner that provides conversational fluency and smooth progress of the task. When considering applications like robot companions which interact with the user in real time, turn-taking and topic flow are also important. This paper describes creation of multi-modal co...
متن کامل